Computer-assisted surgery system and method

ABSTRACT

There is described a system and a method for assisting a user manipulating an object during a surgery, the method comprising: tracking the object in a sterile field in which surgery is being performed at a location using a tracking device which generates tracking data; processing the tracking data using a processing device located outside the sterile field to generate position and orientation information related to the object; and sending the position and orientation information related to the object to a displaying device positioned in the sterile field adjacent to the location at which the surgery is being performed, for display to the user.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of application Ser. No. 15/361,048,filed on Nov. 24, 2016 which claims priority on application Ser. No.12/139,751, filed on Jun. 16, 2008. The present application also claimspriority on U.S. Provisional Patent Application No. 60/944,138, filed onJun. 15, 2007.

TECHNICAL FIELD

The present application relates to computer-assisted surgery systems andmethods, and more particularly to the tracking of objects, likenavigation systems in computer-assisted surgery.

BACKGROUND

Computer-assisted surgery (CAS) systems typically have a tracking deviceto track in space an object involved in a surgery, such as an implant ora surgical instrument. A computer system for outputting informationrelated to the object being tracked is also generally used.Computer-assisted surgery (CAS) systems may display information aboutthe object being tracked on a single displaying device, with or withoutimages acquired by the CAS system.

Some more recent CAS systems involve imageless tracking as opposed tocommonly used image-based systems. The displaying device is howevertypically located outside the surgical sterile field and away from thesurgical table. With this configuration, the medical personnel involvedin the surgery, or present in the room, may view the information beingdisplayed. A user performing or involved the surgery must move his/herhead and/or divert his/her sight from the surgical field of view or thelocation at which the surgery is being performed in order to look at thedisplaying device. Possible involuntary movements of the arms and handsof the user may occur at this moment and create an undesireddisplacement of the object being manipulated or an error in performingactions required during surgery.

The displaying device of typical CAS systems is also for the use ofeveryone in the operation room. The information displayed is the samefor everyone and not necessarily targeted at a specific task of aspecific viewer/user.

SUMMARY

It is therefore an aim of the present disclosure to provide a novelcomputer-assisted surgery system and method that addresses issuesassociated with the prior art.

According to a first embodiment, there is provided a system forassisting a user manipulating an object during a surgery, the systemcomprising: a tracking device for tracking the object in a sterile fieldin which surgery is being performed, and for generating tracking datafor the object; a processing device positioned outside the sterilefield, for generating tracking information related to the object byprocessing the tracking data received from the tracking device; and asterilized displaying device located in a volume within the sterilefield defined as being above a plane of and delimited by an operatingtable and below the shoulders of an operator standing next to a patientlying on the operating table, the displaying device for displaying thetracking information received wirelessly from the processing device tothe user during the surgery, the displaying device having a touch-screenfor interaction with the user through said processing device, thedisplaying device being supported directly by the operating table via asupport extending between the displaying device and the operating table.

According to a second embodiment, there is provided a method forassisting a user in manipulating an object during a surgery, the methodcomprising: tracking the object located in a sterile field at whichsurgery is being performed to generate tracking data, the sterile fieldcomprising a volume defined as being above a plane of and delimited byan operating table and below the shoulders of an operator standing nextto a patient lying on the operating table; processing the tracking datafrom a location remote of the sterile field, to generate trackinginformation related to the object; and wirelessly sending the trackinginformation from the location remote of the sterile field for display inthe volume of the sterile field to the user on a sterilized displayingdevice, during the surgery, the displaying device having a touch-screenfor interaction with the user through said processing device, thedisplaying device being supported directly by the operating table via asupport extending between the displaying device and the operating table.

In the specification, the term “procedure” is meant to refer to aprocedure, a series of actions, steps or operations conducing to an endor a final terminating step, a set of instructions.

In the specification, the term “surgery” is meant to refer to: aprocedure performed on a patient usually with instruments; theperformance of a practical work or of something involving the practicalapplication of principles or procedures; and any operative or manualprocedures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of a computer-assisted surgery system inaccordance with an embodiment;

FIG. 2 is a schematic view of the computer-assisted surgery system ofFIG. 1, placed within a surgery environment with the displaying devicelocated within a sterile field for direct viewing; and

FIG. 3 is a block diagram showing steps of the method in accordance withanother embodiment.

It will be noted that throughout the appended drawings, like featuresare identified by like reference numerals.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 schematically illustrates the various components of the system inaccordance with an embodiment.

The system 10 tracks an object or multiple objects 20 being manipulatedduring surgery (also generally referred to as a manipulative procedure)using markers 22. The system 10 also has displaying devices 30. At leastone of the displaying devices 30 is placed within a sterile field 32.

The system 10 has a subsystem 100 depicted as a group of variouscomponents for illustrative purposes. The components may be regrouped ina unitary device or each components may form separate units.

As illustrated, the subsystem 100 has a processing device 102, atracking device 104, a communications interface 106, and a server 108.

The subsystem 100 optionally has an imaging device 110, an input/outputdevice 112, and a database 114.

The processing device 102 communicates with each one of the displayingdevices 30, which are located in or out of a sterile field of surgery32, via the server 108. The server 108 allows for client-servercommunications, the clients being each one of the displaying devices 30.

The tracking device 104 tracks an object 20 or many objects beingmanipulated by a user or being present within the sterile field 32 whileperforming the surgery. The object 20 can be an implant, a surgicalinstrument, a surgical tool, a reference base anchored to the patient,or any other device involved in the surgery.

The tracking device 104 detects and sends tracking data to theprocessing device 102. The tracking data is indicative of a position andan orientation of the object 20 in space with respect to the trackingdevice 104. The tracking device 104 uses passive or active markers 22 totrack the object 20; these are placed in a known configuration on theobject 20. The known configurations are optionally stored in a memory ofthe processing device 102 or in the optional database 114 for futureanalysis of the tracking data by the processing device 102.

Still referring to FIG. 1, the processing device 102 uses the trackingdata detected by the tracking device 104, the configurations of themarkers 22 and/or any additional information pertaining to the object 20being tracked, such as a computer model of the object 20, to generatethe position and orientation information of the object in space, withrespect to any given reference frame. This information is then sent fordisplay from the processing device to a location in the sterile field32, via the server 108.

The position and orientation information sent to a display device 30 inthe sterile field 32 may be accompanied by representations of bonemodels or of parts of the tools (for example an axis, a tip of a tool).

The position and orientation information sent for display can also takethe form of numerical values, such as angles, distances, differentials,and be treated to represent values associated with the surgery beingperformed.

Any information sent for display in addition to the position andorientation information is optionally processed using the tracking data.For example, representations of bone models or of parts of tools usedduring surgery are adapted or translated into a reference frame toprovide a meaningful spatial display of the information to the user.

The tracking device 104 is any type of tracking device or apparatus thatcan locate markers on an object 20 while the object 20 is manipulated.The location of the object 20 in time can be recorded with respect to aparticular reference frame. Examples of such tracking devices includebut are not limited to: mechanical frames, electro-mechanical devices,acoustic tracking devices, electro-magnetic tracking devices, opticaltracking devices and radio-frequency tracking devices. The markers 22are selected to function according to any one of the above-mentionedtracking technologies (electro-magnetic, optical, radio-frequency andultrasound) or any other type permitting the location of the object 20so as to be able to determine its position and orientation in time withrespect to a given point in space.

Still referring to FIG. 1, the subsystem 100 optionally has an imagingdevice 110 for acquiring images of an area at which the execution of thesurgery is being performed. These images can be stored in the database114 or in any other memory device. When images are acquired, theprocessing device 102 has an optional coordinate translating device (notshown) which then translates the object position and orientationinformation into a coordinate system relative to the images acquired, orwith respect to a reference frame compatible with the acquired images.The acquired images and the object position and orientation informationcan be sent for display, separately or as a unified stream, to any oneof the displaying devices 30 which are located at the sterile field 32.

The imaging device 110 can be any type of imaging system or apparatusfor acquiring images of the area at which surgery is being performed.Images of other areas within the sterile field 32 can also be taken, asdesired. The kinds of imaging device 110 which can be used include butare not limited to: X-ray, fluoroscopy, CT-scan, magnetic resonanceimaging (MRI), ultrasonography.

The optional input/output devices 112 of the system 10 are generallyplaced outside of the sterile field 32. An input/output device 112 canbe a display screen, keyboard, keypad, touch-screen, mouse, or any othercommonly known device allowing the input or the output of informationfrom and to a given user. The data entered via the input/output device112 is optionally stored in the database 114 or any other memory device.

An application 122 is coupled to the processing device 102 forimplementing instructions in assisting a given procedure such assurgery. The application 122 is configured to implement various systemstates which correspond to steps involved in the procedure. Steps orsystem states are, for example, calibrating surgical instruments,performing a bone resection verification, positioning an implant, or anyother type of procedural action.

The database 114 is optionally used to store instructions of theapplication 122 and/or any complementary data pertinent to the givenprocedure or system states. For example, the database 114 can be used tostore: images acquired using the imaging device 110, generic surgeryinformation, specific information pertaining to the various actions tobe accomplished during the procedure, as well as any informationpertaining to a patient, living or deceased body, onto which surgery isperformed. Other data is optionally stored in the database 114, such ascomputer models of tools or body parts which can be used by theprocessing device 102 in the processing of the tracking data, in thegeneration of the position and orientation of the object 20 or in thetreatment of any information for display. The stored data can be in theform of texts, images, animations or in any other kind of format asrelated to the procedure. A user can choose to access the stored data atanytime before, after or during the execution of the procedure.

The complementary data stored in the database 114 is retrieved (refer to“data retrieved” in FIG. 1) and also sent, via the server 108, at thesterile field 32, for display by any one of the displaying devices 30.The displaying devices 30 then display this data to users together withany acquired images, surgery information and the object position andorientation information from the processing device 102.

Still referring to FIG. 1, the displaying devices 30 are generallypositioned adjacent to a location at which the surgery is beingperformed and can be portable, pocket-size computing devices, such aspersonal digital assistance devices (Pda), information appliances,personal communicators, handheld game consoles, ultra-mobile PCs,handheld televisions or any other type of portable displaying devices.

The displaying devices 30 optionally have an input device (not shown),such as a touchscreen, to allow a user interaction such as the entry ofdata. The data entered by the user is optionally stored in the database114 or a database (not shown) of the displaying device 30 and later sentback to the processing device 102. Such entered data is also optionallyused by the processing device 102 to process the tracking data or toprocess any information to be sent for display.

Each displaying device 30 optionally has a user interface and acommunication interface (not shown). The communication interface allowsa communication with the server 108. The user interface and thecommunication interface are optionally implemented as dedicatedinterfaces to a user of a particular displaying device 30. In this way,the information displayed by a particular displaying device 30 istargeted to its user and depends on the nature of the user's role or onthe nature of a specific task performed by the specific user in theexecution of the surgery. The dedicated interfaces of each one of thedisplaying devices 30 may be assigned different communication prioritiesestablishing precedence when establishing communication with theprocessing device 102.

Similarly, the user interface enables the display of the acquired imagesand the object position and orientation information according to apreference of the specific user, or to the nature of the specific user'srole and tasks. Hence, different information is displayed depending onwhether the user is a surgeon, an instrument technician, a nurse or anyother member of the surgical team, for instance. As an example, theserver 108 may concurrently send for display information related to animplant position to a displaying device of a surgeon, and informationrelated to an instrument assembly to a displaying device of aninstrument technician.

FIG. 2 schematically illustrates the system 10 within the context of asurgical environment.

As shown, the sterile field 32 has a displaying device 30 located nextto a body portion onto which the surgery is performed, i.e., adjacent toan area at which surgery is performed. The position of the displayingdevice 30 in the sterile field 32 allows the user 200 (a surgeon or amedical assistant) to access and view the displaying device 30 duringsurgery, and optionally within the field of view of the user whilemanipulating the object 20 or performing the surgery. The displayingdevice 30 can optionally be enclosed in a sterile bag or in anyprotective shield if it is not possible to properly sterilize thedisplaying device 30 itself.

Still referring to FIG. 2, a user 200 who is manually executing aprocedure on a body portion 202 on a surgical table 204, is able to viewthe information displayed on the displaying device 30 without divertinghis/her focus out of the sterile field 32 (or out of his/her surgicalfield of view). In some cases, a simple glance of the eyes of the usertowards the display screen is necessary, as illustrated by the dottedlines in FIG. 2.

Referring to FIGS. 1 and 2, the server 108 is optionally a wirelessbidirectional communications server. Hence, the communications interface106 and the communications interface (not shown) of the displayingdevices 30 are optionally adapted to permit wireless transmission andreception.

Displaying devices 30 such as portable display assistants (PdAs), forexample, can support wireless transmission over radio-frequency andBluetooth standards. Displaying devices 30 are also optionally equippedto support infrared data transmission and reception. Opticalcommunication is thus optionally established between the displayingdevices 30 and the interface 106, which in this case is made to supportsuch communication. Other suitable communication protocols, systems, anddevices are also used.

Wireless communications between the processing device 102 and eachclient displaying device 30 help in preserving the sterile field 32effectively sterile and allow the displacement of the displaying devices30 according to the needs of a particular procedure, the desires of aparticular user, or the nature of a particular user's role or task inthe procedure.

Still referring to FIGS. 1 and 2, multiple displaying devices 30 areoptionally located throughout the surgery room, with at least onedisplaying device 30 placed within the sterile field 32 as illustrated.

When there are many users 200, each having their own dedicateddisplaying device 30, the communication interface 106 optionally has asecurity device (not shown) for assigning priorities to eachcommunication from or to a particular displaying device 30 to or fromthe processing device 102.

For example, the security device implements a scheme which ensures thata surgeon who is performing a surgery has the highest priority. In thisway, the surgeon is able to indicate to the processing device 102whether or not a given step in the surgery is ready to be started orwhether it is completed. In place of the security device, the securityscheme may be ensured by the server 108 and each one of dedicatedinterfaces of the displaying devices 30.

Such a communication scheme is optionally adapted to be used to trainpersonnel and communicate with other users 200 throughout the surgeryroom. For example, the head surgeon's input can provide particulartraining information which is then sent to other displaying devices 30dedicated to assistant users. Each one of the users can also contactanyone located outside the surgery room for external assistance.

The priority scheme is also adaptable to prevent conflicts between thedisplaying devices 30 of various users when simultaneously trying tocommunicate with the processing device 102, and/or to allow or blockcertain communications during critical steps. FIG. 3 shows the varioussteps of the method, in accordance with a second embodiment. Thefollowing description of the steps in FIG. 3 involves numeral referencesas illustrated in FIGS. 1 and 2.

In step 300, a system state is set. The system state corresponds to anaction performed during the surgery by a user manipulating an object forexample. The system state may be set by a user coordinating theprocedure or any other user. Alternatively, the user optionally enters aprocedure name. In any case, the client terminal (such as a displayingdevice 30 as shown in FIG. 1) or an input device (such as device 112 inFIG. 1) can be used for such purposes. This step is optional.

In step 302, complementary data relevant to the set system state isretrieved from a memory device such as database 114 (refer to FIG. 1).This step can be accomplished by the processing device 102 (refer toFIG. 1). Alternatively, a list of steps and other information pertainingto the procedure name entered by the user is retrieved. This step isalso optional.

In step 304, an object manipulated by the user to perform the action istracked in space using a tracking device to generate tracking data. Thisis done in real-time throughout the method, while the surgery is beingperformed. In reference to FIG. 1, the tracking device 104 measures andgenerates tracking data optionally taken using markers 22 placed atknown locations with respect to the object 20.

Now referring back to FIG. 3, in step 306, an image of at least an areaat which the action is performed is optionally acquired. This step canbe accomplished at anytime during the method illustrated,pre-operatively or intra-operatively, although an intra-operative imageacquisition would provide for a real-time imaging of the procedure tothe user. The imaging device used for this step (such as device 110 inFIG. 1) may store the images or data stream acquired on any storingdevice to be used by the system as complementary data.

In step 308, the tracking data received from the tracking device (device104 in FIG. 1) is processed using any processor such as the processingdevice 102 (refer to FIG. 1). The data is processed to determine andgenerate information on a position and an orientation of the object.

In step 310, at least the object position and orientation information issent for display at a sterile field of surgery, to anyone of the variousclient terminals (such as the displaying devices 30 as seen in FIG. 1),via a server.

The processing device such as device 102 in FIG. 1, optionally sendswith the object position and orientation information, complementary datasuch as images or any data relevant for the procedure, includinginformation pertaining to the system state, or any other data stored ina database accessible by the processing device. Once the data sent isreceived by a client terminal (or anyone of the displaying devices 30illustrated in FIG. 1) located in the sterile field 32, the data isdisplayed in the following step 312. The information sent for display toa given client terminal is optionally dedicated information targeted ata specific user of the given client terminal. Such dedicated informationmay depend on the specific user's preferences, tasks, and role.

If complementary data such as images taken during surgery are also sentfor display, the processing device optionally translates the objectposition and orientation information into a coordinate system compatiblewith the images. The translation into a common reference frame canalternatively be performed in step 312, once the data is received at thesterile field.

In step 312, anyone of the client terminals receive the complementarydata and/or object position and orientation information, and displaysthis information. The client terminals can each have a differentinterface and the display of the information and/or complementary datais optionally altered in accordance with the system state, thepreferences, the role and the tasks of a specific user of the clientterminal. For example, if the system state is set for a calibration ofan object, such as various tools or surgical instruments, the clientterminals can display the information and/or complementary data in awindow specifically designed to faciliate the action, optionallyillustrating a given reference frame used.

Step 312 is performed during the surgery, optionally in real-time, andwhile each user executes his/her role and related tasks in order tocomplete the action, using the assistance of the information displayedon his/her client terminal.

In step 314, the user optionally inputs data using his/her clientterminal or any other input/output device. The information entered mayindicate a progression in the execution of the action, may be related tothe procedure itself, the patient or the body upon which surgery isperformed, the object being tracked or any other relevant information.In this step, the data entered by the user is received by the processingdevice of the system via a server.

When there is a plurality of client terminals, each terminals isoptionally assigned a communication priority whereby a precedence isestablished for allowing communication between each one of the clientterminals and the main processing device. Such a communication prioritycan be implemented using any kind of arbitration process, with each oneof the client terminals having a recognizable dedicated interface.

In step 316, when the user-entered data of step 314 indicates that theaction is complete or when the system determines that the action iscompleted from the object position and orientation information and/orthe acquired images, the processing device 102 evaluates whether thereis a next action to be completed. If yes, the method goes to step 318;if NO, the method goes to step 320.

In step 318, the system state is set to a next system statecorresponding to a next action in the procedure. The method thereforereturns to step 302.

In step 320, the system state is set to a Final state; the procedure iscompleted.

Steps 316-320 are replaceable by a step such as step 300, wherein theuser overrides the system by setting a given system state of his/her ownchoosing. Changing the system state can thus be achieved at any timeduring the method. A pre-assigned right to change system states can begiven to a specific user in order to avoid multiple users to vary thestate concurrently. Other authorization schemes can be implemented.

In the above method, it is understood that some of the steps may beperformed concurrently and in a variety of ways. The method involves thecommunication between the processing device 102 (refer to FIG. 1),located in a non-sterile field, and client terminals (refer to thedisplaying devices 30 of FIG. 1) which are located in a sterile field,in order to assist a user in manipulating an object during a procedure.

Additionally, information valuable to the execution of the procedure isoptionally transmitted to anyone of the client terminals such that theusers are well informed of the tasks at hand. The client terminals cancommunicate with the processing device via a server, any informationpertaining to the procedure as entered by a user. The client terminalscan also communicate with one another and be used to contact externalassistance or external systems optionally via the server and theprocessing device.

It is thus understood that several other embodiments of the system andmethod for assisting a procedure can be implemented, and thus fallwithin the scope of the present invention. The embodiments of theinvention described above are therefore intended to be exemplary only.The scope of the invention is intended to be limited solely by the scopeof the appended claims.

1. A system for assisting an object manipulation in a sterile fieldduring a surgery, the system comprising: a tracking device for trackingat least the object relative to a bone while the at least one object andthe bone are in a sterile field in which surgery is being performed, andfor generating tracking data for the object; at least one processingdevice positioned inside or outside of the sterile field, for generatingtracking information related to the object by processing the trackingdata received from the tracking device; a sterilized displaying devicelocated in a volume within the sterile field defined as being above aplane of an operating table, the volume being delimited by a peripheryof the operating table, the displaying device for displaying thetracking information received from the processing device to the operatorduring the surgery, the displaying device having a touch-screen locatedin the volume for interaction with the operator; and a support connectedto the displaying device and positioning the displaying device over theoperating table and into said volume of the sterile field.
 2. The systemas in claim 1, wherein the displaying device is positioned within a samefield of view as a location at which the surgery is being performed froma surgeon's viewpoint, for display of the information to the surgeonduring manipulation of the object.
 3. The system as in claim 1, furthercomprising an imaging device for acquiring images of at least one of theobject and a location at which the surgery is being performed, thedisplaying device displaying the images.
 4. The system as in claim 3,wherein the processing device comprises a coordinate translating devicefor translating the object position and orientation information into acoordinate system relative to the images.
 5. The system as in claim 1,wherein the displaying device is integral to a portable handheldcomputing device in said volume during the surgery.
 6. The system as inclaim 5, wherein the at least one processing device is integral to theportable handheld computing device.
 7. The system as in claim 1, whereinthe displaying device includes multiple displaying devices eachincluding a dedicated user interface, with at least one said displayingdevice in the sterile field.
 8. The system as in claim 7, wherein eachof the displaying devices are dedicated to a specific role.
 9. Thesystem as in claim 7, further including a communications interface forassigning a communication priority between one of the displaying devicesand the processing device, the communication priority being inaccordance with the dedicated user interface involved in thecommunication.
 10. The system as in claim 1, further including adatabase for the at least one processing device, the database storinginformation on at least one of the surgery, the object and the location,the information for being displayed by the displaying device.
 11. Thesystem as in claim 1, wherein the tracking device comprises markersplaced on the object, the markers for being tracked using one of: amechanical frame, an electro-mechanical device, an acoustic device, anelectro-magnetic device, an optical device and a radio-frequency device.12. The system as in claim 1, wherein the at least one processing deviceand the sterilized displaying device have transmitters for wirelesscommunication therebetween.
 13. The system as in claim 1, wherein thesupport is integral to the operating table.
 14. The system as in claim1, wherein the at least one processing device has a transmitter forwireless communication with at least one server of the system foradditional computing to be performed by the at least one server.